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The Mathematical Problem Solving Project (MPSP) had a long-range • 
intention of impacting the material s and procedures of teaching mathe- 
matics to children. So it was clear from the beginning that there 
- woyld be a time when. the total impact of MPSP material s and procedures 
• on children would haye to be measured. • ' 

The firoject goal was to explore ways to improve the problem- 
solving abilities of elementary school children. More specifically, 
I'the project wanted to foster: v 
, (1) imprpVed achievement on nonstandard, process-type problems;. 

{■Z) the use of a r^icher repertoire of skills and strategies, in - 
. p'rolslem solving; and .as a spinoff Hof (1) and (2), 
(J) improved achievement on standard-^ textbook problems. ' ; 
T{J measure the attainment of specific go^ls, it is important to*' 
have instram^pts.'that are familiar and accepted. ,It is also important 
to have instruments which are seVitive to thes^- specifi-c ftals. Dur- 
ing the 1975-76 school 'year, the gsals,of tpe MPSP summative evaluation 
were to identify/existing instruments to measure its goals, to develop 
-instruments where no satisfactory ones existed, and ito" pi lot- test the ^ 
instruments- and procedures "for administering them. This is a report 
of the sel.ectton, development, and pi.lot-testing of instruments .lis'ed 

in the ■1975-s;6 MPSP summative evaluation. . 

^ ... • ' ' , 

Since MPSP was terminated, it has become clear that these efforts 
wi 1 r represent t^e only systematic evaluation of the project. So this 
report will cover both instrument 'selection and testing^and any tenta- 
tive obsery.ations concerning/ the* project that can be bas^ on the data 
from the pilot -testing of these instruments. . It is impqrtant to-'.TJOte 
that the treatment received by the chilidren ihvplved in this study was 



not the systematic application of a developed program but rather ex- 

. ' . . I 

* periences with preliminary materials which were under development. 

While every effort was made/ to ifrsure that the children had exneri- 



ences that were educationally sound, these experiences were in no way 

based on a completely developed program. ' 

B. Iyut/umznt6 and V^odzdix/iU ^ , . ' * 

This report is organized around the jfour instruments selected or 
developed for the summative evaluation and used during the 1975-?6 
school year. The instruments used for the sumniativ,e evaltiation were: 

1/ Studen/ Attftude Questionnaire (SAQ), developed and validated 

• ' ■ • / . ' ' . ^ 

by the MPSP evalliation- staff ; 

■ /■ . • * ■ ■ " ■ ^ ■ , 



2. problems selected from the National Longitudinal Study of 

1 ^ / ' \ 

Matjiemati^s Achjevelnent' (NLSMff); 

3. / the problem-solving subtest of.^the Stanford" Achievement Test 
/ (SAT); ^ * - , . . • ^ 

4. Problem Solving. Survey (Vss ), developed by the MP,^evaluation^ ' 
staff. N * , 

Each of these instruments is briefly discussed below* with respect 
to the procedures used for deve'foping flhe instrument or the process-used 
for selecting the instrument, how the instrument was used, the data 
collected from the instrument, the implications of \he dat^' and what 

was learned about the .instrument. A detailed reportvOn the choice, 

■ . ^ • . ■ ^ ' ■ ■; • • - 

development., and analysts of the instrumentjsj's incliided in the Working v _ 

Paper which begins on page 21. " ,^ 

The children .invol vec( .in the testing reported .here were those students 
in the Oakland Schools that receiv^ all 1;hree MPSP problem-s^ls^'ng mod- 
ules during the 1975-76 school year.. In addition to the experimental classes . 

(i.e., the classes that received the probleW>solving modules), control classes 

9 - ■ 



* were identified.* Both the' experimental , and control classes wetie given 
each of the four instrutifcnt? in t\e fa.ll 1975. before the experimental 

passes used any of the' modules. Each of the four instruments was 
administered again to'ill experimental andjcontrolclasses 'in the spring 
1976. . Al l Instruments. were administered. by* the classroom "teachers. - . 

* 1. Rationale: A year of, exploration witf children and a r'Sview of 
the probVem-solving literature suggested that wilJingrje^s., confidence,/ 
and peifseverance are three factors that^luence problem^solying per- ■ 



• formance^ 'Consequently, the IJIPSP sta^ff felt , it was important to measure 
the impact AOf MPSP on^ students' willingness to solve problems, confi- 
dence in their alJility to- solve problems, and persev^ance ln Attempt- 
ing to obtain a solution. V 7 

The history of paper-and-pencil attilude measur^ent is marked, with 
Successes. Most attitude stuc|^ "result in no signific^ant differ- 
v^^nces; ar)d even when thej^^.are significant differences, they often prove 
: - difficult to interpret. ^MPSP was looking for treatment-specific atti- 
tude change^i.e., specific to -probl em solving) rather than changes in' 
general attitudes. Since the literature on attitude, testing does con-' 
tain some successes! with treatment-specific? instruments, MPSP felt there ' 
was some hope of identifying-treatment-specific attitude .changes. 

2- Development of the SAQ : The details nf thp Hpwoirtpm^nt - 
pilot ^analysis of the SAQ are contained on p^p^ 2^-2^ ^.f the Working Paper 
and in Appendices B, C, 0, and F. Xhe. S/^Q'is a self-ceport,'' paper^ 
.and-pencij instrument'which* contains subscales intended to measure 

^ bee Technical ReporKIH, Chapter A. for a detailed <d'"escription of % ^ 
the experilt)^ntal arid control Classes. . 




willingness, confidence and perseveraj^e. The pilot analysis bf tffe 
SAQ suggested that the SAQ measures some aspects of the affect involved 
in mathematical? problem solving and th.at" the three subscales were dis- 
ti net and. had moderate internal consistency. ^ ' ' / 

^ 3, SAQ D^ta : The. data obtained for the SAQ is reported in Tables 1 , 
2 and 3 for the 4th, 5th and 6th graded.* 'In reporting this data, the ' 
classes were separated into- thirds (lower, middle and upper) -based on 
their scores on th? ^AT. Scdres for the lower third are reported after 
"Lbw,'^far the upper third a^fter '^High," and. for all students af^er 

• >"Whole Class./ Pretest, post-test, and gain scores €(re reported for 
bojUswexperiniental apd control groups; for each of the thneef subscales* 
(wiTlingnes^s, pelrsever^nce and self-confidence) as well as for the whole ' 
test, ' ^ ^: 

At the fourth grade', trends favor the experimental group with a 

smatterins of^signif icant diff^lr^infe At Ijjjg^ifth' grade 

those trends were reversed. At the si xfh grade, results were different 

for different subscales with a. slight over^l edge to the experimental 

group. Looking- at high-ability cWldren and low-abiljty children, one 

notes that high-ability students scored consisten^^Jiigher than lowr 

ability students. ^ " ^ ^ 

• • , \ , ^ ' ' \ ■ 

Ar Analysis of SAQ n^fA^ There^ is no clear trend in the'pretest 

and post-test cortifiarisons for the experimental and control groups. The 

pilot efforts^ with the SAQ indicated that some components of student 

^a^tiydeWare measured by this instrument. However, the data suggests 

that' the MPSP experience did not have'^-a qonsistent impact on the com- 

ponents of student attitude measured by tlje SAQ. All o| the teachers 

in the experimental classes reported positive student attitudes toward 

problem solving related to MP$P,^et no consistent ^ends favoring the 




, Table \ . ' ' 

Comparison of the' Experimental and Control 
Fourth-Grade Classes on. the. SAQ 



Experimen tal 
Post ^ 
(N=l'24) 



GaTn^'~ 
•>(N=121) 



Pre 
(N=65) 



Control 



Post 
(N=73) 



• Gain 
•(N=61 



3/917 
4.673 
4.281 



0.375 
-0.234 
0.068 



,4.5^1*^ 
5.000 - 
4.631 



4; 333 
"5,, 000 



4.519, 



-0.()36 
-0.083 
-0.112 



0.480 

0.017'^ 

6.542 



Perseverance (max; =5) 
Low: §.953 



• T ■ 

. . High: 
Whole Class: 



4.189 
■4. 106 



Self-Confidence (maxi 
Low: 3.875 
Hig^: .^385 
Whole Class: 4.723 



'270 
4.333^ 
4.203 



0.286 
0.188 
0.-^97 



3; 784 
4.308 
3.986 



3.830 
4.538 
4.0&3 



-0.03^ 
0.^231« 
0.077 



1 .761 
0.527 
0.468 



4.194 
5.653 
4.906- 



0.281 
0.313 
0.183 



I 4.189 
5.385 
, 4.391. 



3.619 
5.154 
4.095 



-0.536 - 
-0.231 



0 

6.437** 

2^.765 

6.157 



SAQ ^tal (max. =18) 



Low: 

- ^ Hi gh : 

Whole Cla^s: 



.11.649 . ' 12.400 
14.480; 14.646 
13.169 Vl3.4«2 



0.6>9 
0.333. 
0. 283 



U.647 . IJI.951 
14.43 l|?.692 
13.062 lX793 



^0.560 ' ^".096 
-0.167 0.322 
-0.267. . 1.337 



** p<.05 



■v 



TheN of students represents the. number of students used to compute the ' 
whole i^roup mean. The students used for computing gain ^cof-es were the 
students that completed both the pre^^nd posttests. ' 

'''' Gain = posttesij: scorje- pretest score. ' 

ANCOVA was used to test the difference in gain score-means.' The pretest- 
. for' each istudent was used as the coyariate. • .* 

d ■ . ' ^ ■ - " ' ' ' . ■ ' . ^ 

Low' and high were determined by the Ipwer ancl upper thirds of ttie distri- 
bution of SAT scorv.es for the fourth-grade students. 



J 
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y : •Tat)le 2 • . 

Compari son of the Experimental ^ and Control 
/ Fiftfi-Grade Classes on the SAQ c. 



Experimental 
• Pre^ Post 
^ (N=20^) (N=188) 



Gain^J 
■{N-181) 



Control/ 

HPre Post^ ' GaTrT 

(N=75) (N=74,) (N=70) 



. J- ' .. •. - — ^- r 

WilTingness- 1(max.=6) . ^ 

' Low:^ 3.'769 " 4.136 " Q.301 4.100 

, J High: '4.803 4.864 ' N„169 4.§45 

Whole Class: ' 4.341 • 4:492 ■ -0.151 4.359 



4.364 
4.714 
4.513 



0.462 0.381^ 
0.000 ' 0.556 *^ 
.0.154 -0.003 



^ 1> ' 

Perseverchice (max. =5) 

Low: '3.84^ ' 3.6Q8 

High: ' 4.212 4.117 

Whole Class: 3.991" 3.918 



^.If.-Con-fideRce (max . =7 ) 
Low: ' 3.829 
High: S.485 

.Whole Class: '4.675 



3.618- 

5.533 

4.560 




3.81^ 0.2< 



4.118 
4.013 



0.273 
'0.039 



0.758 
3:831* 
1,609 ' 



^591 
4.449 



3.484 0.03.8, 0.069. 
>5._810 -O.'OOO 0.057; 
- 4.513 , 0.064 0.392 



^SAQ Total (^ax.=18) 

tow: 11.527 



, High: 
Whole Cl^iss: 



14.609 ■ 
13.104 

r 4 



J 

11.521 
14.627 
13.094 



■0.323^ 
0.351 
■O.OlO 



11.379 
14.182: 
12.880 



11 .710 
15.ldo 
13.176 



1 .000 
0.250 
0. 296 



1 .058 

0.080. 

0J60 



> **p<.10 



The-N of students represents the number of student? used 'to compute the 
whole group mean.. The [students used for c^mputihg-gain scores were the 
students that completed both the^ pre- and pasttests. 

Gain = posttest score -pretest score. 

ANCOVA <yas used to 'test the difference in gad n Score means. The pretest 
for each-stu'dent was used as the covariate. ^ :^ ' V' 

Low and Jiigh were determined by^he lower and upper thirds of the distri- 
bution of SAT scores for the fifth-grade students. 
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^ « ' . Table, 3 , ,. .. : ^ . . 

.55f* ^ Comparison of .the Experimental emd Control 

' ' - -Srxth-Graxle Classes on ^the SAQ ■ 



• : ■ Experimental / ^-j- .;; : ' control - '^^ 

Pre" Post , 'Gatn'^': pre . Post — ^Gimr ' c 
'^(N=280): ;CN=270) (N=262) . (N-84r (N=83) {N=83)^ / " 



Willingness (max. =6) ' . ' ■ . ' ' ' A ■ ■ ■ " ^ ,: v ~ 

— > Low:d^ - 4.152 4.038 ' -O'.ljl^ . 4.5.90 4.000 -0.526- 0.111 

; High: ^.SOO 4 J55 f V O.OOO ^, O.OOp 0:97> 

Whole Class: 4.329^ . V^ 4.741- *4.430 /-0.3fl ; t).244 



^ -I 
i/ 

Perseverance 


(max. =5) 








. 4 .128 






■ { 


Low:" 


' 3.626 ; 


3.739 

■ 


• -0. 


115 


^3.810 


-0.297 


0.634 


High:' 


4.130 


4.021 




085. 


':4>300 


[ ,4.214- 


'^).07.1 


0,440 


Whole Class: 


' 3.8^ 


3.833 




048 


4.129 


' 3:949 


-O.aeo 


-^,057 



Sel f-COnf.idence (ma'x . =7 ) 
^ , . Low: . 4;154 14.1134 -0:034 ' 3.900 3.675 -0.f67 1.366 
.Hig.h.r ' 4.673 V 5.695 0.109 . 5«^33 5.964 0^250 



0..696 



0.032 4.721 4.627^ -0.094 1.-376 '- 

SAQ total" (max>18j r • " ° ^' - " ' 

. .' Low: 112.173 12.091:.. 10.321 12.7Vi ; nv700 , -0.857 ;0;828' 

; ' High J 14.633 •• 14.447 -0.022 15.13*3 15.321 0.179. '0.934. 

Whole Class: I3,'i64 1 3.204^ 0.040 13.655 13.-072 -0.583* 0.95fV 





- the N of students represents the number of students used to compute the 
whole group mean. _^The^students used for computing gain scores wer£ the 
students, the^t completed both the pre- and posttests • • '> 



/ 



C 



Gain = posttfest score - pretest Score. ' { - ^ 



ANCpVA was used to .test the 'difference in gain score means. The prefest 
forv^ach student was used as the covarlate, 

^^^'Jow and- high were determined by tfie lower and upper thirds of the distri- 
bution of SAT scores for .the sixth-grade students. 



"^experimental students' were found in> the SAQ daxa/ ThV inconsilstency 
between 'the .teatrtters'^ifeedbac* and the SAQ data* is difficult to inter- 
pret, t,b^eachelrs may have bpen "-judging" studefrtts' attitudes ;on cdm- 
ponetilfe othlr'.than thbs^ mecfsQred by the SAQ. Another "pos- 'U: explana- 
tion is that the SPPff may be too general • an instrument ^ * citti -' v- 
tudes toward^l vin^^prqblems of the typej^'primaViJy useu ^ r.> 

' • ' • . ■ ! ^ " ' ' ' ' ' ■ " ""^^ feail 

: Whatever the reason <for the inconsistency , it -is clear; that furtheV* *W 
■ • ' / . . . ' ■ . * 

efforts are warranted^ to refine or replaqe the SAQ as an instrumerij; for- 



'measuring attitudes toward problem solving jis defined .by MPSP. 

% I 

i!" 1. Rationale : The MPSP staff wanted to ideitify an instrument 

^:opiun^ity and which \ 



that was well-known to the mathematics education^:opiun^ity and which 
measured problem-solving performance on^the kinds of prbWe/s emphasized 
by MPSP.. An extensive review of the literature did not identify any 
such instrument. However, among the problems used in the National 



Longitudinal' Study for Mathematics Achievement (NLSMA) several items 
in. one .subscaVe were similar to the process-^type problems used by MPSP, 
and the NLSMA tests are nationally known and widely accepted. Thr^e 

1tems'^(l-3) were cjiq^en ffom NLSMA more as traditionaXl , textbopk-type 

. # \ 

• problems which would not be threatening to children on\a pretest. Two 

items (4 and 5) were selected as process-type problems. The details • > 
of the selection proij^ss of the five-item, multiple choice NLSMA sub- 
scale appear in the discussion of^the Problem Solving Survey (part'l) 
on page 28 of the Working Paper (see the, NLSMA items in Appendix (a]f. 

2. NLSMA Data : Tables 4, 5 and 6 show the data obtained "from the 
NLSMA.. For fourth and sixth grades, the gains consistently favor^e^ex- 
perimental students over control. Many of the differences were at or 
near significance (p<.05). This picture is even clearer for items 4 
and 5, which are most like, those problems emphasized by MPSP. 

" 13 
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"table 4 ^ 
(?oiTiparison of the Expen'menta,! and Control 
Fouf t^-Grade t-l^ses on NLSMA 



it ems 



Experimental 

Pre^ ' . Post GaTii^ 



We 



Co fitroT 
Tost^ 



. (N=y3) (N=i21). (N=i2'l) in=l9) /■(N=76J ^(^'57 ) 



.V 1>2.3 .(max.=3) ^ ^ 

' . 1.39^ 139 " -0-.22fi , o.OOO 

" Hi^gji:- ^ T.569 . 1.898 ^ 1.^5 1.833 0.083 

- Whole GroupV^^ 1.48'^ li558- . 0.069 ^1.20(^^ 1.165 - 0^5 

4,5 (max. t2) ' . - " . : - ' ., 



3.862* * 
7.188*** 



Lowi 


^."351 


0.788 . 


0,370 


/ 0.382 


0.523 


0.143 


2.661 


^ . High:^ ' 


o>eia 


; 1 .250 


0.419 


0.800 


i.aoo 


0.100 ' 


2,057 . 


Whole Group: 


.'0.579 


1/016 


0.437 


0.516 


0.688 


0.172 


5.324** 


All (max. =5) ■ 




« 




■ ' i\ 






















Low: 


1.750 


1.938 ' 


0.231 


.1.219 


1 .523 


0.111 


. 3.014 ♦ 


X High: 


2.391 


3.146 . I 


0.667 


2.700 


2.750 


tD.lOO 


0.094 


Whole Group: 


2.073 


• 2.603 ✓ 


"0.530 


. 1.729 


" 1 .868 


0.139 


7.761*** 



p<.01 



*** 

** p<.05 
* p<.10 



b 
c 



The N of students represents the n\nber of students used .to compute the 
whole group mean. The students used for computing gain scores were the 
-students that completed both the pre- and posttests. 

Gain = posttest score protest score. 

ANCOVA was uSed to test the differ^ence in gain score means. The pretest 
for each student was used as the covariate. 

Low and high were determined by the lower and upper thirds of the distri- 
bution pf SAT scores for the fourth-grade classes. 
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,, ^ ^ \ : Table 5 
CoftipaMson of the Experimental and CdVitrolv 
Fifth-Grade Classes on NLSMA 



" Items 


Pre^ . 
(N=190) 


Post • V 
' -. (N=188) 


Gain"^ 
(N=181) 


1 ,2,3^ Xmax.=3) 


7— 




-i : ■ 






1.289 


, 0.309 


Hijh: ■ 


2.045 


2.?i)0 ■ 


.0.217 


•Whole Group: 


1.466 


1.686 • 


0.220 



Cgntrol 



■"^*'?re" Post 
(N=71) . (N=77) 



Gain/^ cC 



Oo.966 
^.667 
1.171. 



4.5 (max. =2) 
Low: 
• ; High: 
Whole Group: 



All (max. =^5) 
Low: 
High:' 
Whole Group: 



0.373 


• '0.€25 


0.310 


0.333 


0.841 


1.153 * 


0.386 


■ 0.900 


cr.568 «■ 


0.894 


0.326 


0.611 


1.388 , 


1 .903 


0.638 


1 .333 



2,889 
2.047 



3.373 
'2.585 



0.614 

0.538 

-^k-^ — - 



2.526 

1 .7^9 

If 



*** p<.01 
a 




n/l60 ^ .0.975 
p.m 0.049 
0/57-6 • O.S70> 




0.174 '^3.14^*** 
l./82^ 0.400 / g.l26 . 
0/61/5^ 0.004/.. '9.951*** 



-0.087, 
3/. 364 0.895 
'.351 "0.562 



7.334*** 
0.065 " 
2.088 



"b 
c 



The N of students: represents the number of students used -to compute the 
whole group mean. The students used, for computing gain scores we?^ the ■ 
students that completed both' the Ip^^e- and posttests. 

Gain = posttest score - pretest scpre. " ^ 

ANCOVA was^used to test the diffe??nee i gain score means. The pretest 
for each student was used as the aov^j i tte. ' . 

Low. and high were determined by the lower and upper thirds of the distri- 
b^uti^n of SAT scores for the fifth-grade students. ^ 




n 



" Table 6 ' " v ■ 
* Comparison of the Experimental an(J Control 
- cSixt+f^Grade. Classes on NL^MA • 

— — - 



, Items 



Experimental 



-rrb- 



Control 



P^^^ Gain^ Pre Post ^ Gain r 

^N=276) ,rN=281) •(N-275) (N=84) ' ^5=84) (^='84) ^ 



.j^2.3 (max.^^3) .. 
i» Lpw:^ ^ 
High: 
Whole Group: 



1.3^4^7 "^.709' 

^.287 , "^2,484 

1 .83/7 2.070 

/ _ > 



p.3'84 ', r.200'* '1.51-2 . 0.297 O^.SSS 
0.189, 2.400. , 2.179^-0.^4 • 0.649'" 
0.233 ^ T.m ,1.8^5' 0.09^ . 3.553* 



4.5 (max..=2) 
Low: 
High: 
Whole Group: 



07469 
9(794 
j6.615 



0.824 
r.319 
J. 089 



0.522 
0.474 



^ 0;4'74 
a. 900 
0.631 



.0.619 0.111 
1.074 * 0.148 



o.Soo 



»^ 0.169 



4.024** 
3.619* ' 
12:416*** 



All {max. =5) 
Low: 
High: 
Whole Group: 



/. 



1.811 
3.082 
2.489- 



2.533 
3.819 
3.141 



0.824-- 
' 0.711 
0.652 



,1.658 ,2.122 0,486 3.144* 
3.300 3.259 -0.037 ', '8.275***8 
2.381 2.631 0.250 11.958***- 



*** p<.01 
** p<.05 
* p<.10 



a 



b 
c 



The N of students repr6setjts the- number of studei^s -used to compute the 
whole group mean. - The students^ used for computing gain scores were th^ 
students that completed both the pre- and posttests 



•Gain = posttesC scor4 - pretest score. 



ANCOVA'was used to test the difference in gain score means. The pretest 
for each student was used as the covariate. 

^ ^ow ind high were determined by the lower and upper thirds of the distri- 
^lon of ^ SAT scores for the.sixth-grade students. ' ' 
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The results fo.r the fifth grade are. ldss '^ji^^©:^^ ^^^^^ 
there\is a slight ijrend toward the low-abil;iy ex^n^ntal^ students , 
and there is a. trend ^^ward the [iigh-abil ity control students. Hdwever, 
there is a significant edge , to e^penimental students on problems 4 and 



"5:i Jt shQuld be noted ^tha"t \confl1cting and sometimes surprising results 
■for the fi-P^"^ '^vatV -,+nHents have persist. ' " .naul bo^h 1(<. fu/ma- 
•ti'^e >and summajtiv)^" evaluations. | . 

3. Apal^^ , i s o^l^LSMA Data : The NLSMA subs^ale proved Jo be easy 
to"! administer and easy to s4ore. Problems 1, 2 and 3 did ^jyovide a 
sufficient "amount of success to avoid any difficulties'. in using th^^ 
test* pretest or with control students. The entire NLSMA test has 
ional acceptance,-, and the sobscale. (especially items 4 and '5) has 




na 



face validity with respect to MPSP goals.'. The data collected suggests 
that th^ NLSMA problem! (qspeci^l ly .items 4 and 5) are sensitive to the 

*MPSP treatment. The set of pr^lems seems appropriate for use in any' 
future MPSP evaluations. • 
E. Stanio^d AclUe.vme.yvt Te2t [SAT) 

1. Rationale : The use of a standardized achievement test was de- 
cided upon. for two reasons. First, the project hoped to (^lemonstrate 
that pn a widely accepted instrument, the MPSP treatment did not re- 
sult in a decliTf^ in achievement in traditional l|y-measured aj[;e»s of 

jDiathjematics. Secondly, the project was interested in icnowing if the 
MPSP treatment h^dtSuVf icient impact so that it could be detected on 
a kind of test that ha|^ traditionally proved to be insensitive to treat- 
ment effects. 'By design standardized achievement tests measure those 
skills which are basic to all school currtcula. ' Althougfi many research 
and evaluation studies have reported data frbm ffiese tests, few. have 
found significant differences in means due to treatment. 



In c"hoosing anappraprlate'.achievemeht test; one of * the primary 
^ <irit^ria was that the problejn-solving portion have as much* emphasis 
as popiljle on Pj^cess-type problems. Tb^ Stanford Wievement Test 
was qLsen because'the problem^ solving portion h'ad a spirit that wa. ' 
mo.t. -similar to th..',r ... ^.^^ 

tne^others on^oyr criteria (e^^. . readability)V" The details of the" 
ac|iievement test selection are reported in Appendix A; The SAT itself, 
.jl? Appendix H. . . ' " " ' ^ ^' 

• - 2. ^SAY Data : in ordW to limtt th^^otal testing tinie to ^n 
^acceptable level . only the problfcolving portion of SAT Intermediate 
Levtii I was given. It was administered to experimental and control 
, ,.st^ents on a pretest - posttest basis; ? . - ^ > 

The data shows consistent trends in favor of the experimental clashes 
While the data does not contain consistent ^significant differences, it-^ 
clearly support the (laim that MPSP experiences. do not result in losses 
• 'In more traditional problem-solving skijls. (See fable 7.) 

The SAT was easily administered and seemed to exhibit some sensitiv- 
ity to the MPSP treatment. If teeing time had allowed, it would have 
-been good to administer the computation portion. of the test to support 
a wider claim concerning the lack of negative.side effects (namely, a 
decrease in computational skills) due to MPSP.* 
F. VKobt<ifn So'lvlnQ SuAvzy (?SS] 

1. Rationale : Th^ project wanted ail instrument that was sensitive 
to the MPSP goal*, thatuwould provide some success experiences for most 
children, and that woul^ provide some ~in sight into the, type of problem- 
solving pro<iesses that were actually being used by children. To meet ■ 
these nfeeds the PSS was developed. ?t was 'a three-probleml open-ended , 
test (I.e.. not multiple choice): The first problem on each fortn (four •• 





. . . . ; . Table 7. " . 
• '. Comparison of the Experimental and Control 


SAT Means 


Grade 




Experimental 








< 








u,. ^ 


Pre 


fust 


GlTTn 


4 


students 
. Mean 


•1 'i'i 

'27^64'^ 


116 
jpO.767 


115 
3.203 ' 


66 •' 
22.894 


. 79 
24.291 


64 ■ 

• • 

1.'397 '3.119* 


\ .5 ', 


— " 

^iiu . or 

AS tudents 


cu/ 


\ 

1 OD 


181 


«8. 


71"/ 


' 63 

./ 

1.606' 0.233^ 




Mean 


27 681 


29 605 


1 . 9^4 


27/971 


29.577 


6 


No. of 
students 


286 ( 


\ \ 

266 


262 


72 


82> 


70 • 




Mean 


32.500 


34.5.98 


2.098 


32.028 


32.841 


0.813 ^2.392 


* p< 


.10 




y 








— >—■ : ' — 

« 


^ Gain = postest 


score - 


pretest score. 


■ ■ ■ ;\ 







ANCOVA was used, to test the differences in group means. The pretest score 
for each student was used as the covariate. ' 

Maximum was 40. ^ 




forms ' ach of . three grade I' vei- was a simple, unt-bcep, textbook- 
type wora problem. The second 'problem was^a multiple-step textbook- 
type problem; and the third problem was a pi/ocess-type- problem qf the 
kind -focused on by MPSP. The student was encDiJraged to show all of ^ 
his/hef^work- on'the paper and waj^-given an example oi\what'this meant 
in the Instrument's in'structi'pns: T^e details of the deve>Qpment of .-^ 
the PSS are-coptained in the- Working Paper on pages 28t3Q.v (The PSS is 
referred,to^ there as Part II of the P^blem ^Solvi^g Survey. The PSS-, 
itself V is 'Appendix l-.J . ^ " . , 

2. PSS Data : The PSS was administered in the same way &s the other 

instruments and the results We^re afialyzed in two' ways. First, the tests 

/ " ■ ■ ' ' ' • V 

were evaluated on a "right-wrong" basfs. The data from this evaluation 

was analyzed and reported in the same way as the data for the other 
instruments (see Tables 8, 9 and 10). ■ 
For the correct-answer analysis, the fourth-grade experimental stu- 
dents -had a significantly higher mean gain score than the fourth- 
grarde control students^ (p<.01). At the fiftF- and sixtK-grades, the, 
mean gaip scores for the experimentaV studetits were higher- than the 
/nean gain scores, fof" the control students but the differences were not - 

significant. On the whole, the instrumenlT^id -seem to have som^ensi- 
\ \ ' ' ■ - ■ . . 

At'lvity to MPSP .goals, ' and problem (1) did seem to provide the desired 

^Mccess experience for the" students as ^videnced by the high ^roup mean 

jfor this problem. 

1,:^ A second evaluation of the test was an informal comparison of the 
pr^e- and posttqst papers to -see if the PSS had any potential for reflect- 
inig student problem-solving processes. Growth'in experimental students 
wa^jpbserved with respect to the amouat of work shown, as weVl as with 
re&ct' to the use" of tables to solve problems. The conclusion was that 



■ Table 8 

Comparison of .Experimental ,and Control 
Fourth Grade-Classes on the PSS 



Experimental 



Pve" 
■(N=126) 




Control 



Gain 
fN=112) 
— k— 



Pre 
(N=64) 



Post 
(N=.55) 



Gain, 
(N=52) 



0 



Problem 1 , ^' 

. Low:'^ 0..568 

High: - 0.80Q 

Whole Group: 0.69a, 



V 

0.848 
^756 



0.121 0.500. 0.556 
0.043«* .0.9T7f '0.917 
0.058 



0.6091 -0,492 



0.077 
0.083 



.056^ 
.178 



10.117 2.867 



* » 



Problem 2 

Low: 

Whole Group: 



0,108- 
0:560 ' 
0.365 



0.324 
10.565 

o.hi' 



' .0.242 
^ -0. 043 
^ 0.072 



0.056 0.148, 
. 0? 0.417 
0.*094 ? 0.218 



0.115 
0v149 



2.147 

:,537 



0.124 5.026** 



. Pro.blTem 3 
^ 'Low- 
High: 
Whole Groupr- 



.0.222 ' 0.388^ 
476 . 0.-591 
0.367. 0.501 



0.212 0.161" 0.089? 0.038 7.261*** 
0.096 0.350 0.-467 0.117 .156 

0.134 0.219 0.273' 0.054 3.610* 



Total 

•Low: 
High: 
Whole Group: 



0.897 



1.418 



1:836^ 2.004 
1.430* 1.694 



0.576. 
0.096 
.2f 




0.231 4.486A* 
0^33 1 .769 
^•.m^ 8.711*** 



*** p<.01 
** p<.05 
* p<.10 



J 



the N of attlaents represents the number oT students used to compute the 
w^iole grSup mean. The students used for computing gain scorfts~were the 
students that. completed both the pre- and p&Sttests. / 

Gain 5 posttest score - pretest score. ■ ' 



/i V 



ANCOVl was.use/d to test the di.fference in gain score means. The pr'etest 
for each student -was. used as the cavari ate.. 

Low and high vitere determined by the lower.and upper thirds of. the distri- 
bution of SAT^scores for the fourth-grade students. , . 
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>. Tabled 9 ■ . ' ' ■' J 

_ ^Companison of Experimental and Control ' 

^ « ^ Fif t.h-RraHp riaccoc nn +hQ Dec » 



Fifth-Grade Classes on the PSS 

/■■ 3" Experimental ■ Control ^ 
M Pre . fjost F^aiTr Tre • Pnct ^ r:;.-in 



^ , . ^ . ^ Gain Pre • Post Gain \ 

]W87)^(N=1.68.) (N=16C) (N^6a):_(N=63) (N=61) ^ 



A.. Low:'^' ,0*417. MX^682 ,0.246 0.484 -0.654 ' 0.208 .014 
HTgtr: ^ 0.85d^^t)'.89T 0.07,5.- 0.778.^ 0.889 0.,111 .'I^IOS 
Whole Groups ^ 0 ^0 ^0.786 0.>66 ^ 0 .603 0.825 ' 0.^222 .813 



Problem 2 



T 



Low:. 0.097 0.1 97 . V 0.077 0.161 0.231 0.1:25 ' .143 - 

High: 0.443 0.764 . 0.340 ^3a 0.556 o:222 2.2>54l 

Whole Group: 0.28^ 0.446 0.16'3 0.206 0.381 " ^0.. 175 ^ .'368 

ProblemJ^ ^ « 

Low: 0.091 0.247 0.147 " 0.055 "0,130 0.078 1.222 

• High: ' 0.332 • ' 0.515 0.192 10:39-^.429 0.090 .685 

Whele Group: 0.201 0.393 . 0.192 " 6.161 0.255 0.094 4.-236** 

Total 

Low: 0.605 - 1.126 0.470 0.700 1.014 ' 0.411 .016 

^ High: 1.594 2.170 0.607 1.450 1.874 0.424 .907 

Whole Gr&H^: 1.105 1.625 0.520 , 0.969 1^ 461 " 0.492 .635 



** p<..05 

■ ' '. ' 

a 



, The_N of students represents the. number of students used to compute the 
,whol^ group mean. Jhe students used tpr computing gain scores Wfere the 
"Students that completed both the pre- and posttpsts. ." ' 

^ Gain - posttest score-- pretest score.' . ' * " " 

c ' ' ■ ' ■ " ( ' ' ■ ' ■ ■ , ■ ■ . ■ 

ANCOVA was used to test the difference In c/a1n score means. The pretest 
.for each student was used as the cbVariiite, - ' 

d 

Low, and high were determined by the lower and upper thirds of the distri- 
bution of SAT. scores for the fifth-grade students. ■ ' 
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Table 10 

Comparison o^^ th«^Expetirpental and CorTfrol 

Sixth-Grade Classes on the PSS . 
- . i m 



Experimental 



Pre" 
•(N=285) 



<2M 
•P6 



Control 



St. 

(N=277) 



Gain 
(N=?70) 



Pre 
{N=73: 



Post 
(N=69) 



Gai» \ nC 
{N=158) ^'^ 



^ 
















Problem 1 




u 












-^Low:'^ 


^ 6.547 


0.7^^ 


0.196 


. 0.529 


0.774 


0.226 


1.972 


~High: 


- 0.787 


'0,849 


0.065 


0,750 


0.826 


0.087 


.170 


Whofe Group: . 


,0.667 


0.787 > 


0.120. 


' 0.616 


0.754 


>0.138 


.107 



Problem 2 
















~~^~~1:ow: 


0.170 


"4.267^ 


0.076"^ 


^^^0^088 


0.161 


0.097 


.410 


'High: 


0.447 


0.570 


0.130 


0.458 


0.348 


-0.130 


' 4.702** 


Whole Group: 


0.284 


' 0.426 


0.142 


> b.v233 


0.232 


0.001 


7.511*** 


Problem 3 
















, .Low: 


0.268 


^0.446 


0.173 


• 0.294 


. 0.323 


. 0.032 


.686 


High: " 
Whole Group: 


0.493 
0.389"^ 


0.585 
0.519 


0.107 - 
0.130 \ 


' 0.422 
0.324 * 


0.^58 
^ 0.538 


0.217 
0.214 


.789 • 
.155 " 


Total 
















Low:' 


0.^985 


1.455 


0.444 


' 0.912 


1.258 


0.355' 


.041 


High: 


1.727 


2.004 


0.303 


1.630 


^.832 ' 


0.174 


.738 


Whole Group: 


1.339 


1.732 


0.393 




1.524 


0.351 


1.132 



p<.01 
p<.05 



The N of students' representj^he number of students^^used to compute the 
Whole group mean. The stuc^ents used for cbmputincf gaiji scores were the 
. students that completed both the pre- and posttests'. 

\ • ^ , . , 

Gain = posttest score - pretest score. : ^ 



ANCOVA was used to test 'the difference in gain score means, 
for each student 'was usBd as the covariate. 



The pretest 



Low and high were determined by ^tfhe lower and uoper thirds of^the distri- 
bution of SAT scores for the sixth-^rade stadeprts. . > , < . 



• a careful analysts of PSS papers could reveal group changesJrNthe 

use of problem-solving processeV^especially in those processes which 

• ■ \ , ' ^ 
are^l ikely to be ma|jifest»d on paper (e.g. , making a table, using a 

lis\ drawing a picture, writing an equation, etc^. It was decided- 

thatjn order to make such an analysis meaningful, the PSS wouTcl have 

to be- considerably refined, especially as to the selection of problems 

of type 3. ' , ' > 

c . "/^^ ■ ^ • 

6. -Siurmviy ^ • 

1. SAQ^: I,t iV clear that in or,der to measure attitude chaijge-due 
to the MPSP tj:§atment, mSre^work is needed. 'Informal feedback from' 
teachers and childfen: suggests that the MPSP treatment had a -positive 
impact on student altitudes. It seems that in order to, verify this, 
impacts either the informal feedback procedures will have to Ije^for- ' 
ma>ized (e.g., formal interviews and questionnaires) or considerably 
more -refinement will have to take place in -the SAQ. 

2- NLSMA ; The NLSMA subs cale seems to have been a consistent and 
sensitiv^e Instrument and should probably be considered in any future ' 
.evaiyati on efforts. ' i. ^ * " 

3. iM.' The ^AT has, 1 imited poten^^ 

to the MPSP treatment-:- However, ^' be good to admihister the ' ' 

entire SAT to refute possible claims that the MPSP treatment results 
in dec'reases^in traditional mathematical skills. 

4. PSS:, The; PSS se|ms to provide a non -threatening and sensitive 
approach to measuring MPSP^als. If the process evaluation use of 
PSS Is to be purged, additional attention needs to be given to deter- 
mlnirtg. the processes likely to be used on each problem and to deve.lop- 
1ng reliable rating procedurps, ^ . / . 



5., Implications for MPSP :. ftiven the fact that the^intent^of the 
MPSP treatments durinjg 1975-T976 was to provide forma/iive trial| for 
t^ft prpbliem^-sol ving n!8^^ not tq provide a complete and coherent . 

^•program for the children, and given the fact that the purpose of the 
Nsumpiative evaOjiation effort v^s to pilot-test* instruments and^procedures 
and not to evaluate MPSP, tljie outcfomes were fairly encouraging for the** 



^ntal ^tudentf. 
lor the ex- 



project. Trends in the data generally favored*the experimental ^tuden^^ 
Also, some Instances were nated where, the jnean gain score toi 
perlmental students ^was'significantly higher *han' the' mean gain s^core 
for tbe control students* br\^ process-type problems. On mor^ traditional 
types of problems*, thgjsxperimental ^ students generally performed as 
.well as the'contr;ol students and in some instances outgained the con-- 
trol students. The summative evaluation results related to, student 
attitudes are not clear. - , ^ ' 



,. . MPSP Working Paper 1975/76 ' ' 

Wmation V Tests foe the 5uninHti\e ' Evaluation 

Norman L. Webb • " 

• The evaluation e^orts for the -school year 1975/1976 are divided ' 
into two parts, formative and summative. The fomative evaluation will 
•focus, on each module with the goal of "improving the effectiveness. of " the 
.moduUbs in attaining their objectives. The sumrifativfe' eva-viatiorf wtll ' 
-^focus on the. assessment of chaijges iti the stu^Sents 'and tea.chers over the 
year.. A more detailed explanatibn of the Valuation design is given in 
A Description of an Eval j iation Design for the r^thematical Problem Solving 
Project r " Rationale a ndVoposed Implementation (Lester and Webb, 19'7'5). • 
The purpose of this vcrking 'paper is to delin^te the r-ationale anT^o- 
cedyresused to compile and create the tests for the pre- posttesting for, 
the summcitive evalucition. 

At the June 9. 10. and June" 23. 24.. -1975 MPSP staff meetings ;it was 
agreed by representatives of all three centers to' include the following 
instrumentation in the test battery to be used for the summative evaluatio'n 
. ly^ "problem-solving" portion of a standardized 'test that was s ' 
available for admffristratlon by *tlie Oakland Center.- 
2) *-AWer -and- pencil instrument ^to assess the child's preference 
of. problems and confidence in his a'biltty- to sblve problems 
, 3) A.problem^sOlving survey composed of ;probleiiis at different 
cognitive levels, including problems of the type that the '. 
project considers important. - . , > 

■ 4) A problem-sort task that will be, given to"^ teachers to 

*' ■ ■ . • ^ ■ ' ■ 

assess their perception of the appropriateness of prpblems for^ 
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use in teaching problert solving and their perception of probTems 
which would be preferred by children.* (Note: This sort task^ 
was agreed upon at the June 9, 10 meeting and was admin istereid 
to the< project te^ch^r^jin June 17.) - 
. 5) An interview and observation format that teachers can use to 
evaluate their students' willingness and perseverance with' 
respect to problem solving. * , * 

The dates set for the administration of the tests to the students 
were early October and May. The dates for administering the sort-task 
to the teachers were June 1975. and June 1976 at the first and last in- 
serv^ice meeting of the teachers for the year. It was also agreed upon 
to give tfre testing to a control group of approximately ten classes. 



Standardized test 

.The jDaklartd Center sent to I.U. copies of all the standard>jed tests 
they had in-house and available for administering. The five tests in- 
cluded were: Iowa Tests of Basic Skills (Forms .5 & 6), Stanford Achieve- 
ment Tests (Intermediate Levels I and 11 Form 4), Metropolitan Achievement 
Tests (Intermediate Form F and Elementary Form F), Californ^ Achievement 
Tests , and the SCAT-STEP (Series II, Form FA), These tests were reviewed 
systematically' and judged on their, capability to measure problem-solving 
ability. Very few differences were found among tests. Test 6, Math 
Application, of the Stanford Achievement Test (Intermediate Level I) 
was felt more appropriate for our needs. The selection process is . 
reported in "Evaluation of Standardized Tests" (Charles and Moses, 1975) 
(see, Appendix A)- A copy of Test 6 is included in Appendix H, 
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student Atti/ude Questionnaire ^ ■ . * . 

The SAQ is a paper-pencil questionnaire designed to measure different 
affective aspects of the problem-solving processes of students. The SAQ 
has gone through several changes during its evolution. 

, . Initially, the SAQ >^d three parts. Part I contained fourteen state- 
ments about solving word problems. The student was asked' to indicate 
whether he agreed or disagreed with each "statement. The. items were 
'divided into three scales: willingness to engage in problem-solving \ 
activities (4 items), perseverance during the problem-solVing processes 
(4- items), and self-confidence with respect to problem Solving (6 items)V 
_Ihese items were selected from a list of 136 items coTleSed from several 
.sources including the Childhood Attitude Inventory for Problem Solving 
developed by Covington /1 966) for use with the Productive Thinking, Program . 
I.U. Staff members (total of 10) individually sorted the 136 items into 
four categories. Three of the categories are mentioned above. The . 
fourth Was curiosity aboutxthe solving of a problem'.!>Wtthii each category 

'V " ,. . . ■ ' ' ■ .' ' ". ■ • "■■ 

the r.atQi;-s ranked/the item? Taticordin^. to how Well .thprj tern jexpressed thfr " 
attitude. Vltems jn the tuf iosi.ty category tfitfm'ot-^jv^'^nyf^ohesiiv^ 
or the appearance of forming one-attribute^ofi'thj^-^^^ 
category was dropped. - ' 

Items that all could agree upon to fit in a category and ranked high 
within that category by several people were used in Part I of the SAQ. , 
Care was taken to ensure that within one category positive and negative 
items were evenly repres^ted. 

Part II was designed to have students respond to different types" of • 
problems with respect to the three categories of will ingness , perseverance. 
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and self-confidence. Four problems were included. Two were word probldhs 
like those found jn textbooks j^ne was a process -puzzle type problem (a 
problem from the project^s^robTem^^eclc)"r^^ a group-project, 

problem. Three 'statements, reflecting each of the three categories, were 
listed after each problem. The students were to respond whether they 
agreed or disagreed with the statement with respect to the given probl.em. 

Part III was designed to assess the student's preference as to the 
type of problems he/she would solve. Two sets of three problems were 
givea. Each set represented a range of |)roblem types (textbook, process- 
puzzles, and group-project). Fpr each set of problems, the students were 
asked to select the problem they would, most like to solve and then the 
problem they would next like to solve if given a choice. A third ques- 
tion then asked, without any restrictions, how many of the .problems would 
they like to try. 

The SAQ with three parts (s«e Appendix B) was administered to three 
classes of students representing a range in grade levels. In addition, - 
Individual interviews were conducted as students worked the SAQ., Two 
modes of responding were used: dichotomous (Agree/Disagree) and a five- 
point scale (Agrep-^ 2 3 4 5 Disagree). Appendix C contains tables 
comparing me ar^ between the two forms for each item in Part I. As a re- . 
suit i)f these tryouts a few rewording changes were made in Part I. More 
quest ioK5/were raised about the effectiveness of Barts II and III. Re- < 
sponses in Part II varied very little. Because of this and the difficulty 
that some students experienced in reading the problems. Part ^I was 
dropped. 
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. In Part III *4udents selected the problems they would most like to 
work. The reasons for their choices varied. From the individual inter- 
views,- the mo^ capable students would select problems on the basis of 
attributes related to the structure of 'the proMem, such as "it was the 
•easiest" or "I knew I could work it." The other students would select - 
- problems for superficial reasons such as "it was the first one-" or 
liked the story." It appeared"tha,t knowing only the student's problem" 
preference may be misleading. To know why the student selected a partic- 
ular problem was also needed. At the end of the year the student may 
select the same problem for different reasons. The change in reasons 
could be important. Part III was modified to include a list of reasons 
that the student could use to indicate why he selected a particular 
problem (see Appendix D). C - - 

The revised Student Attitude Questionnaire contained two parts. 
Part I had fourteen statements for the student to respond "agree" or 
"disagree" (using the five-point scale). Part II as-ked the student to 



select from a set' of three, the one problem he would most lik^o work, 
on and to-give the reason why he selected the probferii. The SAQ in this 
form was reviewed^by representatives of all three centers at the evalua- 
tion meeting of the Wagon Wheel Conference. 

The evaluation working session parjbicipants agreed that it w^s still 
uvlear Exactly what Part II, problem preference (formerly Part III), 
would measure in its present form.. For ^h is reason. Part II wis deleted 
from the SAQ. I A point was made that if dimensions of problems were 
identified that appeared to be important to problem solving then these 
could be used to form contrasting pairs of problems. For example, a 
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proJbleJ with two* conditions could be compared with a problem with several 
conditions. A problem with ^n associated picture could be compared to 
one without a picture. TheUtudent would be asked to select one of t;he 
pair of problems that he would like to solve.' This could be used to 
measure the student's preferences for certain problem dimensions. 

It was decided to' increase Part I to include twenty items. With 
the elimination of PantWl, more time could be devoted to trying to 
measure more meaningfully the three attributes of willingness, perseverance 
and self-confidence. The present form of Part I was thoug'ht to be. too 
impersonal in that students were asked to respond^er statements made by 
pther students. It was felt that our information would be more valid 
if the items were changed to true-false and the student asked if the 
statement'represented how he felt. 

.The I.U. Center generated seme more items. Willingness was broken 
down into three dimensions "gutting", cooperating, and liking. "Gutting" 
represented the- wil 1 ingness to try something without regard to its' 
difficulty.. Cooperating represented, the willingness to go along with 
sbmething— it is the thing to do. Liking represented the willingness to 
engage in something because it is fun — I like to do it. Two items, one 
positive and. pne negative, were- included for each of these three dimen- 
sions of willingness. ^ 

Perseverance was separated into three dimensions: obtain right 
answer i not premature closure, and stick-to-itiveness . One kind of ^ 
perseverance is sticking with the task until the right answer is founcff . 
Another is sticking with the task until just any^ answer is found, that 
is, not stopping work before some closure has been reached. The third 
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is stick-to-itiveness, worjcihg on a problem for a long tiififand not giving 
up rtght away. This last dimensiiOn is not dependent on getting ajjy tyje 
of, an answer. As with the willingness stale.ftwo items (positive and - 
negative) were selected for each of these three dimensions of perseverance. 
. Self-confidence was divided into three dimensions: . belief to' succeed, 
■comparison to others, and "guts. > Bel ief to succeed is the idea that 
"I am a good, problem solver an^^will succeed most of the time." Compari-, 



son^o others represents the. person's self-con^d|^ce with respect to 
^i^/her peers— I am better or worse compared to other students. The 
iuts of self-confidence relates to the difficulty of the situation— 
I can solve hard problems'. Foup items (two positive' and two negative) 
were included under belief to succeed. Two Ste^s were included under 
each of the other two dimensions. ■ ' > A 

The final 'SAQ appears. as Appendix E. Table 1 lists each Item in. 
the scale andjtbe-dimension of the scale that $he, item, represents. 



TABLE 1 



> Item Numbers-^for Each Dimension of Each Attitude Scale 



Scale 

^ 


Dimension 


Positive 
Item. 


Negative 
Item f 




Gutting ^ 
Cooperating 


5 • 


15 


Will ingness 


3 


14 


< 


Liking 


17 


2 




Right Answer 


16 , 


4 


Perseverance 


Not Premature Closure 




1 




Stick-to-itivenes§, 




18 




Belief to Succeed 


9 , 20 


12, 19 


Self-confidence 


Comparison ) 


11 


6 




Guts 


r 13 


7 
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. Problem-Solving Survey ^ ' . 

• The draft of the problem-solving survey brought to the Wagon^j/heel 
evaluation workina session consisted of two parts. Part .1 was the NLSMA 
scale X023 which was, given by NLSMA to the X population in the fall of 
. the fourth grade. X023^ontains five questions in a multiple choice 
format and was ratied by the NLSMA staff to be at the analysis l/eve^T - ' 
This scale appeared to be appropriate for the needs of MPS P for two 
reasons. First, two of the problems (4, 5) require processes similar 
to those which thp project is focusing on. One prob^eiTr(^) relates- to 

using a table and the otf^r (4) to finding a pattern. Second, the 

• 1.1 

multiple-choice format gives "students an opportunity to respond. Some 
concern was expressed about having problems so difficult that some stiK- 
dents could not do any of the problems and would become very frustrated. 
In a multiple-choice test, a student has jL^d^ce and can always ^uess 
; if he/she is not sure of the answers. Othe.r issues that" supported the 
use of the NLSMA . scales Were the ease of grading multiple. clioice items 

/and. the availability of data from the^ administration ofi the scale to a 

' ■ i' '~ 

national population. 

Part ir of the survey (open-ended) consists of problems for the 
students to work, showing all of ti|eir work on paper. In the first draft 
of the survey each student is given two s^ets of three problems. From 
each set the student selects one problem to work.' The rationale for 
giving the students a choice of problems was that they would feel 
better if they could s&^ect a problem that they could relate to instead 
of being forced to solve a giveit problem. 
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1,1^11^?'^''^"°'^ *° Conference, the two part? of the problem- 

sio|jnng survey were tried- out with ten students, some individually and 
softi^ in groups of two. The problems were tried both, as multiple-choice 
and as open-ended questions. Most of the students could do at<^least 
one problem on the NLSMA scale X023. For problems 2 -and 3, the students 
had ttie idea and could find the answer to the open-ended question. How- 
ever, when asked to find a number sentence which could and could not 'be 
used to solve the problems, they had trouble. In Part II, the better 
students liked to have a choice of problems and were able to handle the^ 
situation. Other students, as with Part II of the SAQ, selected problems 
for cosmetic reasons and, in. some cases, did not read more than one 
problem. ' • ' 

The members of the evaluation'working session approved the use of 
the NLSM^ scale, X023, for Part I of the problem-solving survey.' How- 
ever, the members did n6t feel vthat giving the student's a choice of 
pr|)6lems to, solve in Part II/W^ the most: efficient way.' *:instead, it , • \ 
was decided to construct foil^ forms for each grade level. Each form 
was to have three .problems which represented a range of difficulty ■ ^ 
levels,. Each stuclent, then, would work on all of the problems in the 
'form she/he was given, showing her/his work and answers on the' paper. ' 

A set of approximately forty problems was* compiled after the 
Wagon Wheel Confev^ence representing three types of' problems: textbook • 
word problems that^ had one or two-steps; problems (like those found in 
a textbook) that required more than two steps and that were not directly 

translatable into a number sentence; and the process-puzzle pVicfelem's ^ 

Si • 

like those in the project's' problem bank. The Utter type of problem 



was selectee! to insure that problems' were included which could be < 

solved by using different processes: .guess & test, tables, listing, ' \ 

and searching for a pattern. 

The'l.U. staff selected problems from the set of forty to construct 

the four forms for each of the three grade levels. Each form contained 

three problems one of each type. In some cases the sam^e problem was 

ftsed at different grade levels. The first problem was selected to 

'' , • * ' * • 

(ensure almost certain success. The instructions to the students indi- 

' . - >i> . 

cated 4hat they were to write down how they solved the problems. One 

Qf the grade 4 forrfis (4Aj) was given to one class to see if the instructions 
were clear- arid how the students would respond to the directions. The 
response was very favorable. Almost every student wrot^ down his computa- 
tions and some idea about working the problem. Every student\olved at 
least one pro^tem and 21 out of 23 students solved the first fjroblem 
correctly. 

PaH I (the 'NlSMA 1tejTisro,f the final prpbl em solving survey appears . 
as Appendix G, and Part IF(the open-ended problems) appears as Apoendix I. 

r 
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